A Collective Approach to Harness Idle Resources of End Nodes

We propose a collective approach for harnessing the idle resources (cpu, storage, and bandwidth) of nodes (e.g., home desktops) distributed across the Internet. Instead of a purely peer-to-peer (P2P) approach, we organize participating nodes to act collectively using collective managers (CMs). Participating nodes provide idle resources to CMs, which unify these resources to run meaningful distributed services for external clients. We do not assume altruistic users or employ a barterbased incentive model; instead, participating nodes provide resources to CMs for long durations and are compensated in proportion to their contribution. In this thesis we discuss the challenges faced by collective systems, present a design that addresses these challenges, and study the effect of selfish nodes. We believe that the collective service model is a useful alternative to the dominant pure P2P and centralized work queue models. It provides more effective utilization of idle resources, has a more meaningful economic model, and is better suited for building legal and commercial distributed services. We demonstrate the value of our work by building three distributed services using the collective approach. These services are: a collective content distrbution service, a collective data backup service, and a high-performance computing service on top of the collective overlay.